Method and apparatus for processing medical image data in a network environment

ABSTRACT

The present invention is directed to the processing of medical image data in a network environment as a network service. In an embodiment of the invention, a network image fusion service is provided that is configured to receive a plurality of image data and to execute at least one image fusion processes in both software and hardware. The image fusion process logic has been divided between instructions to be executed in software and logic to be executed in hardware. The software instructions may be executed by a processor within the network element or in a processor associated with network service. The hardware logic is executed in a reconfigurable programmable logic device such as a Field Programmable Gate Array (FPGA). The network service may be included within the network element, be separate from a network element but associated therewith, or may be external to the network and accessible by a plurality of network elements.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention generally relates to image processing systems and inparticular to a method and apparatus for processing medical image datain a network environment.

2. Description of the Related Art

Medical diagnostic imaging allows radiologists to perform diagnosis ofmany types of injury and disease by imaging various internal body parts.For example, radiologists utilize diagnostic imaging to visualize organsin the abdomen, the chest cavity, the brain and central nervous system,the musculoskeletal system, and other body parts. Diagnostic imaging maybe used to detect potential cancer abnormalities; bone densitometry;joint, bone, or soft tissue injuries; and many other types of diseases.Presently, diagnostic imaging includes many different imaging modalitiessuch as x-rays, ultrasound, computed tomography, magnetic resonanceimaging, and nuclear medicine to name but a few.

Traditionally, almost all diagnostic imaging was film based. An imagewas recorded on a physical piece of film that had to be developed,provided to the physician for viewing, reviewed by the physician, andrecorded and stored in an archive. Often there was a significant timedelay between the taking of the image and the physician reviewing theimage. In addition, the storage of film images required a large physicalspace and associated record keeping. If a physician needed to refer to apatient's stored records, the film images needed to be physically found,retrieved, and provided to the physician. Often there was a significanttime delay in this process as well.

To address these issues, diagnostic imaging technology has advanced andmedical diagnostic imaging has shifted from a film based system to adigitally based system in which diagnostic images are recorded,transferred, viewed, and stored electronically. Several types of digitalimaging modalities, such as computed tomography or magnetic resonanceimaging generate a large number of images during each diagnosticexamination that are then combined to form a three-dimensional volumeimage. One examination can generate between twenty and one hundredimages or more. The processing of the large number of image files torender a single fused image is time consuming, so that a techniciantypically performs the image fusion prior to the fused image beingprovided to the radiologist for review. In some instances, the imageformed by the technician is not exactly what the radiologist needed andthe determination is made that other images are necessary. The timedelay between taking the image and the review by the doctor orradiologist may thus be considerable. Thus, the patient may be requiredto return at a later time for more images to be taken, or the patientmay be required to wait until the radiologist has reviewed the imagesand made a determination as to whether more images are needed. In eithercase, the patient is required to be subjected to further inconvenience.

SUMMARY OF THE INVENTION

A method and apparatus for the processing medical image data in anetwork environment is provided to reduce the amount of time required toprocess images for example to combine or fuse a plurality of image datainto a single image data. According to an embodiment of the presentinvention, a network service is provided that is configured to receive aplurality of image data and to execute one or more image fusionprocesses in both software and hardware. The logic for each image fusionprocess may be divided between instructions that are to be executed insoftware and logic that is to be executed in hardware. The softwareinstructions may be executed either by a processor within the networkelement or in a processor associated with network service. The hardwarelogic is executed in a reconfigurable programmable logic device such asa Field Programmable Gate Array (FPGA). The network service may beincluded within the network element, be separate from a network elementbut associated therewith, for example, as a blade server, or may beexternal to the network and accessible by a plurality of networkelements. The process logic and hardware parameters may be stored withthe network service components or may be stored external to the networkas well.

BRIEF DESCRIPTION OF THE DRAWINGS

Aspects of the present invention are pointed out with particularity inthe appended claims. The present invention is illustrated by way ofexample in the following drawings in which like references indicatesimilar elements. The following drawings disclose various embodiments ofthe present invention for purposes of illustration only and are notintended to limit the scope of the invention. For purposes of clarity,not every component may be labeled in every figure. In the figures:

FIG. 1 is a functional block diagram of a network environmentincorporating an embodiment of the present invention;

FIG. 2 is a flow chart illustrating a process according to an embodimentof the present invention;

FIG. 3 is a functional block diagram of a network element according toan embodiment of the invention located; and

FIG. 4 is a functional block diagram of a network element and associatednetwork service module according to an embodiment of the presentinvention.

DETAILED DESCRIPTION

The following detailed description sets forth numerous specific detailsto provide a thorough understanding of the invention. However, thoseskilled in the art will appreciate that the invention may be practicedwithout these specific details. In other instances, well-known methods,procedures, components, protocols, processes, and circuits have not beendescribe in detail so as not to obscure the invention.

As described in greater detail below, a method and apparatus is providedfor processing medical image data in a network environment to form imagedata from a plurality of image data files in conjunction with thetransmission of the plurality of image data files across a network. Inone embodiment of the invention, a network service is provided andconfigured to execute one portion of an image fusion process in softwareand to execute another portion of the image fusion process in areconfigurable and reprogrammable hardware device, such as a FieldProgrammable Gate Array (FPGA). The division of the process logicbetween software and hardware is a function of the of process step, thelevel of complexity of the process, the speed of the processor used toexecute the software instruction, the desirability of providing forfuture upgrades, and other system requirements with respect to speed andmemory storage.

According to an embodiment of the present invention, the network imagefusion service is provided in conjunction with a network element that isconfigured to receive a plurality of medical image data, retrieve theprocess software instructions and the hardware logic files for at leastone image fusion process, execute the image fusion process both insoftware instructions and in hardware logic and to provide the desiredfused image as an output. In the event that there is more than one imagefusion process to be executed, the network service executes theplurality of image fusion processes in a predetermined order that may bea stored value or set by an operator. By dividing the execution of theimage fusion process logic between software code and hardware logic, andexecuting these on the network with a network service in conjunctionwith a network element, the present invention described herein is ableto shift responsibility for the image fusion processing from medicalequipment used to display the images or an image archive system, to thenetwork service. This allows a user to request a particular image and toreceive the image in a shorter period of time than images formed usingonly software implemented processes or images processed by thereviewing/reporting workstation or image archive system.

Although one aspect of the present invention described herein isdirected toward a network service configured to fuse multiple medicalimages into a single medical image, the invention described is notlimited in this manner as other forms of image processing may be done aswell, such as edge detection, filtering, morphing, and other types ofmedical image processing.

In the embodiments that follow, the network image fusion service isdescribed as being used in conjunction with a network which may be, forexample, an enterprise network that may be deployed in a medicalfacility or other facility. Examples of typical networks may include aradiological information system (RIS) or a hospital information system(HIS). However, the network may also be a more extensive network such asa wide area network (WAN), a metro area network (MAN), or a publicnetwork such as the Internet.

As used herein, image data means the pixel data and other data that areused to render a single image and a plurality of image data means pixeldata and other data representative of a plurality of images that may becombined to create one or more combined images. Image data for a singleimage and a plurality of image data representing a plurality of imagesmay be stored in one or more image data files depending on the systemconfiguration and requirements. Also as used herein, hardware logicfiles may include without limitation, scripts, setup files, tool lineupfiles, and other such files that implement the process logic in areprogrammable device such as an FPGA.

As depicted in FIG. 1, one or more imaging modalities 102 are configuredto generate two or more image data that are to be fused into a singleimage data. The imaging modalities may include, without limitation, anx-ray system, a computer tomography system, an ultrasound system, amagnetic resonance imaging system, or a nuclear medicine system. Othermodalities may similarly be used and the invention is not limited tothese particular modalities. The image modalities 102 transfer the imagedata to a desired destination for storage, processing, or display via anetwork 104 that is made up of one or more network elements 106.

The image data provided by the image modalities 102 can be provided toan image archive system 108 for storage or to a reviewing/reportingworkstation 110 for display to be reviewed by a radiologist. Accordingto an embodiment of the present invention, an image fusion networkservice is deployed on the network to provide the image fusion ofmedical files transferred over the network 104. A network image fusionservice module 112 that provides the network image fusion servicefunctionality may be located, for example, on one or more of the networkelements 106 configured to communicate on the network.

In the embodiments described herein the network image fusion module 112is associated with a network element 106 that is configured to form apart of the network 104 described above. For example and withoutlimitation, the network element 106 may be a router, bridge, gateway,content switch, or other type of network device, capable of hosting thenetwork image fusion service. Alternatively, as will be explained inmore detail below, the network image fusion module may be a blade serverthat is provided with an interface to the associated network element106, but that has its own processor, memory, and reconfigurablereprogrammable logic device, e.g., a FPGA, and is capable of executingboth the software instructions and the FPGA logic.

FIG. 2 is a flow chart that depicts a process for performing real timeimage fusion according to an embodiment of the invention. Initially, arequest for an image created from a plurality of image data supplied bya data source is generated (202). Optionally, the request may beaccompanied by a desired order of execution of one or more image fusionprocesses. The data source may be an image archive system or an imagingmodality. The data source provides the plurality of image data andplaces the plurality of image data on the network (204). The pluralityof image data are received by the network element associated with thereal time image fusion service (206). The network service in conjunctionwith the network element requests the software instructions and hardwarelogic files, e.g., the FPGA parameters, for each of the real time imagefusion process from a storage location (208). The storage locationprovides the software instruction and the hardware logic files for therequested process (210). The network element receives the softwareinstruction and the hardware logic files for the requested process andprovides these to the network image fusion service (212). The networkimage fusion service, in conjunction with the network element, loads andexecutes the software instructions and the hardware logic files in thereprogrammable hardware for the image fusion process (214). If anotherprocess is to be run, (216), the network image fusion service, inconjunction with the network element, loads and executes the softwareinstructions and the hardware logic files in the reprogrammable hardwarefor the image fusion process to be executed (214). If no more processesare to be run, the fused image data on the network (218).

In the event that there is more than one image fusion process, the realtime fusion service storage location may provide the softwareinstructions and hardware logic files for each process individually, inwhich case the process iterates through process steps (208) to (214) foreach process. Alternatively, if there is more than one image fusionprocess, the storage location may provide the software instructions andthe hardware logic files for all processes at the same time. In thisevent, the real time image fusion service stores the softwareinstructions and hardware logic files and iterates at step (214) untilall the processes have been executed thereby.

FIG. 3 depicts a network element 106 according to an embodiment of thepresent invention. In particular, the network element generally includesa processor 302, which includes control logic 304, and a memory 306. Theprocessor 302, control logic 304 and memory 306 provide thefunctionality and control of the network element 106. The networkelement 106 also includes one or more network data ports 308 that enablethe network element 106 to be connected to the network 104. A switchfabric 310 under the control of the processor 302, is provided tointerconnect the network data ports 308 and to direct packets betweenthe network ports 308. The switch fabric 310 may be supported by apacket queue 312 that is configured to temporarily store packets orother protocol data units prior to transmission on the network 104 orbefore being processed by processor 302.

The network element 106 may also include one or more subsystems underthe control of the processor 302 and control logic 304. For example, ifthe network element is configured to make routing decisions for datapackets on the network, routing software 314 and routing tables 316containing routing information may be provided to enable the networkelement to route data packets and other protocol data units on thenetwork. Other subsystems may include for example a protocol subsystemthat includes a protocol stack 318 that is configured to store data andlogic to enable the network element to participate in protocol exchangeson the network. The network element 106 may also include a securitysubsystem 320 that may include an authentication module 322 that isconfigured to store authentication information to authenticate users,devices, network connections, or a combination thereof. The securitysubsystem 320 may further include an authorization module 324 that isconfigured to provide authorization information to prevent unauthorizedaccess to the network, the network element, or both. The securitysubsystem 320 may also include an accounting module 326 that isconfigured to enable accounting entries to be established for sessionson the network, the network element, or both.

As depicted in FIG. 3, the network image fusion service module 112 isconfigured to perform the image fusion service described above withrespect to FIGS. 1 and 2. In the illustrated embodiments, a networkimage fusion service module 112 is coupled to the processor 302 andmemory 306. The network image fusion service module 112 includes aprogrammable reconfigurable logic device 328 and a memory 330. Thenetwork image fusion service module 112 may optionally include thestorage location 114 of the image fusion process(s) softwareinstructions and hardware logic files. Alternatively, the storagelocation image 114 of the fusion process(s) software instructions andhardware logic file(s) may be external to the network element 106 andcoupled to the network element via one of the ports 308 or via aseparate port 332.

The image fusion service module 112 may be implemented on a networkelement 106 as illustrated in FIG. 3 or may be implemented on a separatecomputer platform so that the fusion service module can also beimplemented separate from the network element 106 as illustrated in FIG.4. In the embodiment illustrated in FIG. 4, the image fusion servicemodule 112 includes a processor 402 in addition to the componentsdescribed above with respect to FIG. 3. In this embodiment, the imagefusion service module 112 can be implemented, for example, as a bladeserver external to the associated network element 106, but coupledthereto to enable the blade server to receive and transmit messages viathe network 104 and to make use of the processor and memory and otherresources available within the network element 106.

The methods described herein may be implemented as a set of programlogic that are stored in a computer readable memory within the networkelement or accessible thereto, and executed on one or more processorswithin the network element. However, it will be apparent to a skilledartisan that all logic and methods described herein can be embodiedusing discrete components, integrated circuitry such as an ASIC, PLA,PAL, FPGA, or microprocessor, a state machine, or any other deviceincluding any combination thereof. Programmable logic can be fixedtemporarily or permanently in a tangible medium such as one of thevariety of read-only memory (ROM) chips, a computer memory, a disk, orother storage medium. Programmable logic can also be fixed in a computerdata signal embodied in a carrier wave, allowing the programmable logicto be transmitted over an interface such as a computer bus orcommunication network. All such embodiments are intended to fall withinthe scope of the present invention.

It should be appreciated that other variations to and modifications ofthe above-described method and system for transferring and compressingmedical image data may be made without departing from the inventiveconcepts described herein. Accordingly, the invention should not beviewed as limited except by the scope and spirit of the appended claims.

1. A method for processing medical image data, the method comprising thesteps of: receiving, by a network service, medical image data via anetwork element; retrieving, by the network service, software codes andhardware logic files corresponding to an image processing process;configuring, by the network service, a reconfigurable and reprogrammablelogic hardware in response to the hardware logic files; and executing,by the network service, the image processing process softwareinstructions and hardware logic files.
 2. The method of claim 1 whereinthe step of receiving the medical image data comprises: receiving theplurality of medical image data from a data source.
 3. The method ofclaim 2 wherein data source is an image archive system.
 4. The method ofclaim 2 wherein the data source is an image modality.
 5. The method ofclaim 1 wherein the reconfigurable and reprogrammable logic hardware isa programmable logic device.
 6. The method of claim 5 wherein theprogrammable logic device is a Field Programmable Gate Array (FPGA). 7.The method of claim 1 wherein the step of reconfiguring includes:reconfiguring, by a processor in the network element, the reconfigurableand reprogrammable logic hardware
 8. The method of claim 1 wherein thestep of executing comprises: loading the software instructions into amemory for execution by a network element processor; executing, undercontrol of the network element processor. the image fusion process logicboth in software by the network element processor and in hardware by thereconfigurable and reprogrammable logic hardware in the network service.9. The method of claim 1, wherein the step of executing comprises:loading the software instructions into a memory for execution by anetwork service processor; executing, under control of the networkservice processor. the image fusion process logic both in software bythe network element processor and in hardware by the reconfigurable andreprogrammable logic hardware in the network service.
 10. The method ofclaim 1, further comprising the step of providing the processed medicalimage data as an output.
 11. The method of claim 1, wherein the imageprocessing process includes an image fusion process, wherein the medicalimage data includes a plurality of medical image data and the imagefusion process is operative to combined the plurality of medical imagedata into a single medical image data.
 12. The method of claim 1,wherein the image processing process is an edge detection process. 13.The method of claim 1, wherein the image processing process is an imagefiltering process.
 14. A network system for processing medical imagedata, the system comprising: a network element including a processor anda memory, the network element coupled to a network and configured tosend and receive data via the network; a network service coupled to thenetwork element, the network service configured to: receive medicalimage data via the network element; retrieve software codes and hardwarelogic files corresponding to an image processing process; configure areconfigurable and reprogrammable logic hardware in response to thehardware logic files, the reconfigurable and reprogrammable logichardware being included in a network service; execute by the networkservice the image processing process steps in both software codes andhardware; and to provide as an output the fused image.
 15. The system ofclaim 14, wherein the network service is operative to receive themedical image data from a data source.
 16. The system of claim 15,wherein the data source is an image archive system.
 17. The system ofclaim 15, wherein the data source is an image modality.
 18. The systemof claim 14 wherein the network element includes a network processor andmemory, and wherein the network service is operative to load thesoftware instructions into the processor for execution by the networkprocessor.
 19. The system of claim 14 wherein the network serviceincludes a network service processor and memory both being coupled tothe reconfigurable and reprogrammable logic hardware, and wherein thenetwork service processor receives the software instructions andhardware parameters and the reconfiguration and execution of logic bythe reconfigurable and reprogrammable logic hardware is directed by theservice processor.
 20. The system of claim 14 wherein the reconfigurableand reprogrammable logic hardware is a Field Programmable Gate Array(FPGA).